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As  required  by  the  fiscal  year  1989  National  Defense  Authorization  Act 
(P.L.  100-456),  we  evaluated  the  performance  of  the  Army’s  line-of-sight 
forward  heavy  air  defense  system  during  operational  testing.  The  act 
requires  that  the  Secretary  of  Defense  certify  that  the  system  meets  or 
exceeds  the  Army’s  operational  test  performance  criteria  before  the  Sec¬ 
retary  of  the  Army  obligates  procurement  funds  after  fiscal  year  1989. 
Accordingly,  our  objective  was  to  determine  whether  the  system  had 
adequately  demonstrated  its  operational  suitability  and  operational 
effectiveness.  We  are  reporting  on  the  classified  test  results  in  a  sepa¬ 
rate  document. 


Results  in  Brief 


Operational  testing  did  not  demonstrate  that  the  line-of-sight  forward 
heavy  air  defense  system  was  operationally  suitable.  The  system  fell  far 
short  of  its  availability  requirements,  in  large  part  because  many  system 
components  were  unreliable.  In  addition,  other  suitability  measures 
were  either  not  tested  or  not  met. 


The  system  did  not  meet  a  number  of  effectiveness  requirements  during 
operational  testing.  These  include  the  individual  fire  unit  criterion  for 
destroying  threat  aircraft  within  a  specified  engagement  area  and  detec¬ 
tion  requirements.  In  addition,  we  believe  that  platoon-level  and 
response  time  requirements  for  the  system — although  met — may  have 
been  set  too  low. 


The  Army  has  acknowledged  the  system’s  reliability  problems;  however, 
it  believes  that  the  system  demonstrated  sufficient  operational  effec¬ 
tiveness  to  support  continuation  of  the  program.  In  August  1990,  the 
Army  decided  to  defer  production  of  the  system  for  2  years.  During  that 
time,  the  Army  hopes  to  resolve  the  system’s  reliability  problems  and 
demonstrate  its  suitability  for  combat. 
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Background 


Because  the  system  has  not  demonstrated  that  it  can  meet  established 
requirements,  by  law  the  Secretary  of  the  Army  may  not  obligate  any 
procurement  funding  after  fiscal  year  1989  until  the  system  meets  or 
exceeds  the  Army’s  operational  test  performance  criteria. 


In  November  1987,  the  Army  chose  its  current  line-of-sight  forward 
heavy  air  defense  system,  trade-named  the  “Air  Defense  Antitank 
System”  (adats),  to  provide  needed  air  defense  to  the  maneuver  force 
(see  fig.  1).  adats  was  selected  partly  because  it  was  in  production  in 
Canada  and  therefore  could  move  quickly  into  production  to  satisfy 
Army  requirements.  An  early  Army  estimate  of  the  system’s  unit  cost 
was  about  $1 1 .2  million. 
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adats  is  one  of  five  components  in  the  Army’s  planned  Forward  Area 
Air  Defense  System,  which  together  are  expected  to  meet  the  Army's 
forward  air  defense  needs,  adats'  mission  is  to  defend  tanks  and 
infantry  fighting  vehicles.  The  Army  expects  adats  to  significantly 
enhance  its  current  air  defense  capability  because  of  the  system's 
extended  ranges  and  its  ability  to  operate  in  adverse  environments.  The 
system  consists  of  a  launcher  and  eight  missiles  that  are  guided  by  a 
laser  beam  and  mounted  on  a  modified  Bradley  Fighting  Vehicle  chassis. 
The  system  must  be  positioned  so  that  it  has  a  line  of  sight  to  enemy 
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aircraft.  A  shoot-on-the-move  capability  was  not  part  of  the  system’s 
requirement. 


The  Army  conducted  two  phases  of  operational  testing,  which  were 
completed  in  May  1990.  In  the  first,  individual  adats  units  fired  live  mis¬ 
siles  against  drone  aircraft  at  White  Sands  Missile  Range,  New  Mexico. 

In  the  second  phase,  the  Army  conducted  a  series  of  tests  that  replicated 
ground  battles  at  Fort  Hunter-Liggett,  California,  during  which  firings 
by  air  and  ground  weapon  systems  were  simulated  by  lasers. 


ADATS’  Operational 
Availability  Is  Not 
Adequate  to  Complete 
Its  Mission 


Operational  availability  measures  the  proportion  of  time  a  system  will 
be  available  to  successfully  conduct  assigned  missions.  Such  availability 
depends  on  the  reliability  of  component  parts  and  the  time  it  takes  to 
maintain  them,  to  acquire  needed  spare  parts,  and  to  repair  or  replace 
broken  component  parts. 

The  Army’s  operational  tests  showed  that  the  adats  system  did  not  meet 
its  requirements  for  overall  operational  availability  or  for  the  reliability 
of  individual  components.  Further,  we  believe  that  the  test  results  over¬ 
state  actual  availability  because  the  Army  excluded  important  data 
from  its  calculations.  In  addition,  adats  exceeded  the  maintenance  time 
allotted  for  each  hour  of  the  system’s  operation.  Moreover,  the  Army 
did  not  design  these  tests  to  collect  certain  information  necessary  to 
fully  evaluate  the  Army’s  ability  to  maintain  or  logistically  support 
ADATS. 


Availability  and 
Reliability  Requirements 
Were  Not  Met 


The  operational  availability  requirement  for  fielding  adats  is  71  percent. 
For  operational  testing,  the  Army  set  an  interim  availability  require¬ 
ment  of  55  percent  and  reported  an  adjusted  availability  of  39  percent 
during  the  tests. 


The  system  also  fell  far  short  of  its  requirements  for  the  reliability  of 
individual  components.  Reliability  is  measured  by  the  average  time 
between  operational  mission  failures,  which  include  crew  performance- 
related  failures,  and  between  failures  solely  caused  by  equipment.  In 
both  cases,  requirements  were  set  and  tested  for  the  overall  fire  unit  and 
for  the  missile  weapon  subsystem. 

For  the  fire  unit,  the  average  time  required  between  operational  mission 
failures  is  60  hours  at  the  time  of  fielding  and  38  hours  for  the  opera¬ 
tional  tests.  However,  the  system  demonstrated  an  ability  to  operate  for 
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Army  Results  Overstate 
Availability 


only  about  9  hours  before  it  failed.  For  the  weapon  subsystem,  the 
fielding  requirement  is  92  hours,  and  the  interim  test  requirement  is 
56  hours.  The  subsystem  demonstrated  about  8  hours.  This  performance 
also  represents  a  decline  from  1987  tests  in  which  the  demonstrated 
average  time  between  operational  mission  failures  was  16  hours  for  the 
fire  unit  and  17  hours  for  the  weapon  subsystem. 

The  system’s  performance,  as  measured  by  the  average  time  between 
equipment  failures,  has  also  worsened  despite  Army  efforts  to  improve 
it.  The  demonstrated  average  time  between  equipment  failures 
decreased  from  17  hours  in  1987  to  about  12  hours  for  the  fire  unit  and 
to  1 1  hours  for  the  weapon  subsystem  during  the  operational  tests. 


In  calculating  an  operational  availability  of  39  percent,  the  Army 
excluded  some  critical  test  data  and  made  certain  erroneous  assump¬ 
tions.  We  did  not  quantify  the  cumulative  effect  of  all  the  problems  that 
we  found  in  the  Army’s  calculation.  However,  on  the  basis  of  the  effects 
that  we  did  calculate  and  the  discrepancies  outlined  below,  we  believe 
that  the  actual  availability  of  adats  is  lower  than  reported  by  the  Army. 

For  example,  the  Army’s  calculations  did  not  include  any  test  data  from 
one  of  two  test  phases — the  missile  firings.  This  phase  was  critical 
because  it  was  the  only  time  during  testing  that  the  entire  system  was 
actually  in  operation.  Our  calculations,  using  the  methodology  the  Army 
applied  for  the  force-on-force  test  results,  show  an  operational  availa¬ 
bility  of  33  percent  for  the  missile-firing  phase. 

In  addition,  although  the  system  requirements  stipulate  that  all 
unscheduled  maintenance  time  be  included  in  test  calculations,  the 
Army  included  only  corrective  maintenance  time  associated  with  opera¬ 
tional  mission  failures.  For  example,  as  a  result  of  a  malfunction  with 
the  system’s  radar — a  mission-critical  piece  of  equipment — about  6 
hours  were  needed  to  repair  the  subsystem  over  a  4-day  period.  Because 
the  malfunction  was  not  scored  as  an  operational  mission  failure,  how¬ 
ever,  the  maintenance  time  was  not  included  in  the  calculation.  Like¬ 
wise,  unscheduled  maintenance  time  needed  to  replace  a  unit  that 
affects  the  turret’s  movement  was  not  included.  In  total,  the  Army  per¬ 
formed  145  hours  of  unscheduled  maintenance  that  were  not  included  in 
the  force-on-force  availability  calculation. 
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Mission-Essential  Failures 
Not  Adequately  Defined 


Limited  Maintenance 
Capability  Demonstrated 
During  Testing 


Army  requirements  documents  define  the  “mission-essential”  functions 
that  an  air  defense  system  must  perform  to  successfully  defend  the 
heavy  maneuver  force.  These  include,  for  example,  the  ability  to  shoot, 
move,  and  communicate.  Identifying  the  component  failures  that  result 
in  a  loss  of  mission-essential  functions  is  critical  because  test  calcula¬ 
tions  of  operational  availability  include  the  failure  of  and  logistical  sup¬ 
port  needed  for  only  those  component  failures.  The  Army  did  not  clearly 
define  these  operational  mission  failures  until  tests  were  underway  and 
after  partial  scoring  results  were  already  known.  We  believe  that  these 
definitions  should  have  been  approved  before  testing  began.  Waiting 
until  after  some  scores  were  known  provides  the  appearance  of 
affecting  the  outcome  of  test  results  by  deciding  the  criteria  to  be  used 
after  the  fact. 

We  believe  that  the  definitions  used  to  evaluate  adats’  operational  avail¬ 
ability  inappropriately  eliminated  the  failure  of  some  components  that 
were  essential  for  adats  to  perform  its  mission.  For  example,  after  the 
tests  were  underway,  the  Army  decided  not  to  consider  the  failures  of 
the  laser  range  finder  to  be  operational  mission  failures,  even  though  (1) 
soldiers  used  it  as  an  integral  part  of  the  test  missile’s  firing  sequence; 
(2)  the  Army  determined  that  the  range  finder’s  faulty  performance  was 
responsible  for  missed  targets  during  the  missile-firing  phase  of  opera¬ 
tional  testing;  and  (3)  test  procedures  required  that  the  range  finder,  as 
a  key  component,  be  operational  for  the  start  of  each  missile-firing  test. 
If  these  data  had  been  included  in  the  Army’s  calculations,  the  number 
of  operational  mission  failures  would  have  been  higher,  and  adats’  oper¬ 
ational  availability  would  have  been  lower. 

Finally,  built-in  test  equipment  messages,  designed  to  alert  the  adats 
crew  to  component  failures,  were  not  always  treated  as  indications  of 
operational  mission  failures,  as  is  required  by  test  procedures.  Doing  so 
would  further  reduce  the  system’s  availability  rate. 


The  adats  schedule  calls  for  contractor  maintenance  support  above  the 
organizational  level,  or  that  level  immediately  above  the  crew’s  mainte¬ 
nance  responsibility,  until  the  system  is  fielded  outside  of  the  United 
States.  Because  there  are  legal  prohibitions  on  contractor  involvement  in 


Page  6 


GAO /NSLAIV9 1-51  Air  Defense  Antitank  System 


B-240822 


Logistics  Support  Not 
Tested 


operational  testing,1  the  Army  decided  to  evaluate  the  maintenance  pro¬ 
vided  only  at  the  organizational  level  during  operational  tests.  The  test 
results  showed  a  limited  Army  capability  to  maintain  adats  at  the  orga¬ 
nizational  level.  Army  maintenance  personnel  demonstrated  their  capa¬ 
bility  to  meet  one  of  the  two  maintenance-related  requirements,  but  fell 
far  short  of  meeting  the  other. 

The  average  time  to  correct  equipment  failures  at  the  organizational 
level — or  the  mean  time  to  repair — was  0.62  hours,  a  figure  that  was 
well  within  the  requirement  of  1.5  hours.  However,  not  all  organiza¬ 
tional  maintenance  tasks  were  tested;  criteria  for  the  time  associated 
with  each  task  have  not  been  established;  and  the  allocation  of  tasks 
among  various  levels  of  maintenance  was  not  based  on  firm  require¬ 
ments.  Therefore,  the  value  of  this  information  is  limited. 

The  second  measure  of  performance — people  and  time  allotted  to  orga¬ 
nizational  level  maintenance  per  system  operating  hour,  or  maintenance 
ratio — was  set  at  0.094  maintenance  hours  per  system  operating  hour. 
This  requirement  was  not  met.  The  system  demonstrated  a  ratio  of  0.56, 
or,  in  other  words,  the  maintenance  support  needed  for  each  hour  of 
system  operation  was  almost  500  percent  more  than  allowed  by  the 
requirement.  This  poor  performance  resulted  in  part  from  the  system’s 
numerous  reliability  failures. 

Without  clearly  defined  tasks  and  associated  repair  times  identified  at 
all  maintenance  levels,  the  Army  will  not  have  a  clear  understanding  of 
the  maintenance  needed  to  support  the  system. 


The  Army  performed  a  qualitative  assessment  rather  than  a  test  of  the 
logistics  support  that  will  be  required  for  adats  because  it  has  not  yet 
determined  adats  logistics  support  requirements.  The  test  calculations 
assumed  that  waiting  for  spare  parts,  maintenance  personnel,  and  trans¬ 
portation  services  involved  in  correcting  operational  mission  failures 
would  take  an  average  of  14  hours.  The  actual  time  may  change  after 
the  Army  conducts  its  logistics  support  analysis.  Any  increase  in  time 
spent  waiting  for  parts,  personnel,  or  other  services  would  decrease 
adats’  availability.  In  similar,  previous  testing  of  an  air  defense  weapon, 
the  Sergeant  York,  a  23-hour  delay  was  assumed. 


‘Contractor  involvement  during  operational  testing  of  a  major  defense  acquisition  system  is  prohib¬ 
ited  by  statute  unless  the  contractor  is  to  be  involved  in  the  operation,  maintenance,  and  support  of 
the  system  when  it  is  deployed  in  combat. 
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ADATS’  Operational 
Effectiveness  Is 
Questionable 


Operational  Effectiveness 
Requirement  May  Be 
Understated 


The  Army  reported  that  adats  demonstrated  superior  mission  perform¬ 
ance  for  conditions  under  which  it  was  tested  during  operational  testing. 
The  system  did  not,  however,  meet  its  requirements  for  an  individual 
fire  unit’s  performance  against  targets  within  a  certain  engagement 
boundary,  nor  for  the  detection  of  threat  aircraft.  We  are  reporting  on 
the  specific  results  in  a  separate  classified  document. 

In  addition,  we  question  the  system’s  effectiveness  because 

some  requirements  might  have  been  set  too  low  and 
the  operational  tests  did  not  demonstrate  all  of  the  required  perform¬ 
ance  capabilities. 

The  tests  were  based  on  two  measures  of  operational  effectiveness — the 
ability  of  an  individual  fire  unit  to  destroy  a  target  within  certain 
boundaries  and  the  ability  of  a  platoon  of  four  fire  units  to  defeat  enemy 
air  attacks.  The  fire  unit  criterion  is  contained  in  the  Army’s  required 
operational  capability  statement  for  the  line-of-sight  forward  heavy  air 
defense  system,  which  predates  the  selection  of  adats.  Test  officials 
from  the  Army  and  the  Office  of  the  Secretary  of  Defense  determined 
that  a  more  operationally  accurate  measure  of  effectiveness  would  be  at 
the  platoon  level  against  all  attacking  targets  regardless  of  boundaries. 
Therefore,  they  derived  a  second  criterion  from  the  cost  and  operational 
effectiveness  analysis  that  had  been  performed  for  the  Forward  Area 
Air  Defense  System  program. 

For  both  the  fire  unit  and  platoon-level  criteria,  requirements  for  the 
system’s  effectiveness  were  set  against  fixed-wing  and  rotary-wing  air¬ 
craft.  A  number  of  additional  performance  measures,  some  of  which  had 
quantitative  requirements,  also  were  identified. 


Although  adats  met  its  overall  platoon-level  effectiveness  requirement, 
we  believe  the  requirement  might  have  been  set  too  low  to  accurately 
reflect  the  conditions  expected  when  the  system  is  fielded.  The  cost  and 
operational  effectiveness  analysis,  from  which  the  criterion  was 
derived,  contained  several  assumptions,  such  as  the  following,  that  we 
believe  understate  the  requirements  for  adats’  effectiveness: 

The  analysis  assumed  that  adats  would  always  be  available,  that  is,  that 
its  availability  would  equal  100  percent.  This  availability  rate  is  greater 
than  the  requirement  of  71  percent  and  much  greater  than  ADATS’ 
demonstrated  performance  of  39  percent  or  less. 
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Some  Required 
Performance  Capabilities 
Were  Not  Demonstrated 


•  The  simulated  battle  on  which  the  minimum  platoon  requirement  was 
based  showed  that  artillery  made  a  significant  contribution  to  the  out¬ 
come  of  the  battle,  yet  artillery  was  not  used  in  the  operational  tests. 
Further,  the  analysis  assumed  that  adats  had  a  gun,  which  contributed 
to  its  performance  in  battle.  The  Army  currently  has  no  firm  plans  to 
produce  a  gun  for  adats. 

•  The  analysis  did  not  address  a  number  of  threats  to  adats. 

Another  indication  that  the  adats  platoon-level  effectiveness  require¬ 
ment  may  be  too  low  is  that  it  can  destroy  as  many  or  more  friendly 
aircraft  than  threat  aircraft  and  still  meet  the  requirement.  In  addition, 
the  requirement  against  primary  targets  is  not  only  lower  than  the 
requirement  against  secondary  targets,  but  also  significantly  lower  than 
that  required  for  an  individual  fire  unit.  There  is  no  direct  correlation 
between  the  criteria  contained  in  the  required  operational  capability 
documents  and  used  for  fire  units  and  the  lower  criteria  developed  for 
operational  testing  for  platoons. 


The  Army  had  planned  to  use  the  operational  tests  to  assess  adats’  per¬ 
formance  under  a  variety  of  realistic  conditions.  However,  because  of 
safety  considerations  and  other  factors,  the  tests  did  not  sufficiently 
demonstrate  some  capabilities  critical  to  performance  on  the  battlefield. 
Some  capabilities  were  not  tested,  while  others  were  tested  but  did  not 
meet  requirements. 

Shortcomings  were  especially  critical  in  the  missile-firing  phase.  The 
Army  allocated  1 1  missiles  for  the  operational  tests  and  actually  fired 
only  9.  These  limited  tests  were  not  adequate  to  demonstrate  all 
required  performance  capabilities. 

Test  and  performance  problems  are  summarized  as  follows: 

•  During  live  missile  firings,  only  one  successful  shot  was  fired  against  a 
hovering  helicopter.  Our  analysis  of  test  data  indicates,  however,  that 
the  helicopter  never  hovered,  flew  too  high,  and  remained  exposed  too 
long  to  truly  represent  the  threat. 

•  The  Army  did  not  measure  the  system’s  performance  against  maneu¬ 
vering  targets  during  either  the  live  missile  firings  or  the  ground  battle 
testing,  although  there  is  a  requirement  for  such  a  capability. 

•  Testing  under  adverse  weather  conditions  did  not  take  place. 
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•  Information  on  the  system’s  capability  at  night  was  not  sufficient 
because  safety  concerns  prevented  threat  helicopters  from  using  real¬ 
istic  tactics. 

•  Force-on-force  tests  of  the  laser  range  finder  were  not  conducted  under 
realistic  battle  conditions.  However,  data  gathered  during  the  missile¬ 
firing  phase  of  testing  showed  that  the  range  finder’s  poor  performance 
was  responsible  for  one  missed  firing  and  ont  missed  engagement.  In 
addition,  according  to  Army  officials,  data  on  the  range  finder  from  the 
force-on-force  trials  could  not  be  analyzed. 

•  The  ranges  allowed  during  the  ground  battles  for  simulated  missile  fir¬ 
ings  exceeded  those  that  had  been  demonstrated  under  realistic  test  con¬ 
ditions.  Because  the  laser  range  finder  could  not  be  used  during  most  of 
the  force-on-force  testing,  target  range  data  was  sometimes  unavailable 
to  the  adats  crew.  To  compensate,  adats  was  allowed  to  fire  at  ranges  up 
to  almost  twice  as  far  as  those  demonstrated  in  the  missile-firing  phase 
and  was  given  credit  for  these  simulated  kills.  Army  officials  believe, 
however,  that  technical  test  firings  are  sufficient  to  demonstrate  range 
capabilities. 

•  The  requirement  for  firing  a  certain  number  of  missiles  per  time  period 
was  tested  and  demonstrated  in  only  one  live  missile  firing. 

•  The  requirement  that  crew  members  reload  and  rearm  the  system’s  mis¬ 
siles  was  tested  but  not  achieved.  Although  soldiers  met  the  time 
requirement,  the  loaded  missiles  were  not  available  for  firing  in  suffi¬ 
cient  quantities. 


The  Army’s  New 
Program  Proposal  May 
Not  Overcome 
Problems 


The  Army  has  decided  to  defer  adats  production,  has  developed  a  new 
schedule,  and  is  developing  associated  quantity  and  cost  data  for  the 
program.  Production  is  being  postponed  for  at  least  2  years;  quantities 
will  decrease;  and  initial  estimates  of  unit  costs  show  an  increase  of  over 
$3.6  million  per  unit.  The  Army  hopes  to  resolve  the  system’s  reliability 
problems  and  demonstrate  its  operational  suitability  through  a  follow- 
on  evaluation  before  the  next  production  decision.  However,  the  Army 
has  not  yet  determined  the  full  scope  of  testing  in  the  proposed  schedule 
and  may  not  conduct  additional  operational  testing  before  the  full-rate 
production  decision. 


Congressional  Funding 
Actions 


The  Army  received  $54.8  million  in  research,  development,  test,  and 
evaluation  funding  for  fiscal  year  1990.  It  also  requested,  and  Congress 
appropriated,  $208.9  million  in  procurement  funding  that  cannot  be  obli¬ 
gated  until  the  Secretary  of  Defense  certifies  that  adats  successfully 
completed  operational  testing.  For  fiscal  year  1991,  the  Army  requested 
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$271.8  million  in  procurement  funding  and  $9.1  million  in  research, 
development,  test,  and  evaluation  funds  for  ADATS-related  product 
improvements. 

Since  the  completion  of  our  review,  the  conference  committees  on 
defense  authorization  and  on  defense  appropriations  have  denied  the 
Army’s  request  for  procurement  funding  in  fiscal  year  1991.  The  confer¬ 
ence  committee  on  defense  appropriations  also  rescinded  vhose  procure¬ 
ment  funds  that  had  been  appropriated  in  fiscal  year  1990.2  However, 
the  Army  requested  and  funds  were  authorised  and  appropriated  in  the 
amount  of  $92  million  for  continued  research,  development,  test,  and 
evaluation  of  adats.  Because  of  continuing  congressional  concerns  about 
adats  performance,  the  conference  committee  report  on  defense  appro¬ 
priations  restricts  half  of  that  funding  until  other  studies  are  completed. 


The  Department  of  Defense  (dod)  provided  official  comments  on  the  con¬ 
tents  of  our  draft  report.  (See  app.  I.)  It  did  not  address  the  specific 
findings  in  the  report,  but  recognized  the  seriousness  of  them,  dod  also 
pointed  out  that  further  testing  and  dod  reviews  of  the  restructured 
adats  program  will  take  place  prior  to  a  full-rate  production  decision. 

In  light  of  the  severity  of  reliability,  availability,  and  maintainability 
problems,  escalating  system  costs,  and  the  compressed  2-year  schedule 
within  which  the  Army  plans  to  identify  and  resolve  problems  and  test 
fixes,  the  Chairman  of  the  House  Armed  Services  Committee  has 
requested  that  we  review  the  Army’s  new  adats  program.  In  addition, 
the  report  of  the  House  Committee  on  Appropriations  on  the  fiscal  year 
1991  defense  budget  requests  that  we  evaluate  the  planned  testing 
program. 


Appendix  II  sets  forth  the  objectives,  scope,  and  methodology  of  this 
review.  As  arranged  with  your  office,  unless  you  publicly  announce  its 
contents  earlier,  we  plan  no  further  distribution  of  this  report  until  30 
days  from  its  issue  date.  At  that  time,  we  will  send  copies  to  the 
Chairmen  of  the  Senate  Committees  on  Appropriations  and  on  Govern¬ 
mental  Affairs  and  the  House  Committees  on  Appropriations  and  on 
Government  Operations,  the  Secretaries  of  Defense  and  the  Army,  and 


2These  funds  were  identified  for  potential  reductions  in  our  report  Defense  Budget:  Potential  Reduc¬ 
tions  to  the  Army  and  Navy  Missile  Programs  (GAO/NS1AD-90-302&R,  Sept.  I960). 
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Agency  Comments  and 
Our  Evaluation 


B- 240822 


the  Director  of  the  Office  of  Management  and  Budget.  Copies  will  also  be 
made  available  to  other  interested  parties  on  request. 


Please  call  me  at  (202)  275-4141  if  you  or  your  staff  have  any  questions. 
Major  contributors  to  the  report  are  listed  in  Appendix  III. 


Richard  Davis 
Director,  Army  Issues 
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Comments  From  the  Department  of  Defense 


OFFICE  OF  THE  DIRECTOR  OF 
DEFENSE  RESEARCH  AND  ENGINEERING 


WASHINGTON,  DC  20301 


27  SEP  1390 


Mr.  Frank  C.  Conahan 
Assistant  Comptroller  General 
National  Security  and 

International  Affairs  Division 
O.S.  General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Conahan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the 
General  Accounting  Office  (GAO)  draft  report,  “Army  Acquisition: 
Operational  Testing  of  the  Air  Defense  Antitank  System  Reveals 
Serious  Weaknesses",  dated  13  September  1990  (GAO  Code  393345/ 
OSD  Case  8461). 

The  DoD  recognizes  your  concern  with  the  seriousness  of  the 
findings  specified  in  the  draft  report.  It  is  not  possible  to 
address  each  finding  specifically  because  the  final  Army  test 
data  is  not  yet  available.  Upon  receipt  of  that  data  we  will 
examine  it  in  detail. 

The  Army  has  approved  a  restructured  program  to  correct 
reliability  problems  with  the  system.  Additional  testing  is  an 
integral  part  of  the  restructured  program.  We  have  scheduled 
Conventional  Systems  Committee  (CSC)  and  Defense  Acquisition 
Board  (DAB)  reviews  on  31  October  1990,  and  16  November  1990, 
respectively,  to  assess  the  viability  of  the  restructured  Line 
of  Sight-Forward-Heavy  program.  Further  testing  as  well  as 
cost,  performance,  schedule,  and  supportability  are  to  be 
addressed  in  that  process. 

Assuming  Congressional  authorization  and  appropriation  of 
the  necessary  funds,  authority  to  proceed  into  full  rate 
production  will  be  contingent  upon  approval  by  the  Defense 
Acquisition  Board.  Based  upon  successful  completion  of  the  RAM 
Maturation  Phase,  which  includes  a  Follow-On  Evaluation 
(Operational  Test),  certification  will  be  provided  per  the 
fiscal  year  1989  Defense  Authorization  Act  (P.L.  100-456)  that 
the  system  is  operationally  suitable  and  effective. 


Sincerely, 


Frank  Kendall 

Acting  Deputy  Director 

(Tactical  Warfare  Programs) 
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As  required  by  the  fiscal  year  1989  National  Defense  Authorization  Act 
(P.L.  100-456),  we  evaluated  the  performance  of  the  Air  Defense  Anti¬ 
tank  System  (adats),  the  Army’s  line-of-sight  forward  heavy  air  defense 
system,  during  operational  testing.1  Our  objective  was  to  determine 
whether  the  system  had  adequately  demonstrated  its  suitability  and 
effectiveness  for  combat. 

Final  Army  and  Office  of  the  Secretary  of  Defense  reports  on  the  results 
of  adats’  operational  testing  were  not  available  when  we  completed  our 
review.  As  a  result,  this  report  is  based  on  our  analysis  of  interim  test 
results  and  other  program-related  documents,  our  observations  of  the 
operational  tests,  and  our  discussions  with  Army  and  Office  of  the  Sec¬ 
retary  of  Defense  officials.  Our  calculations  are  based  on  the  programs 
and  the  data  base  of  the  Army’s  Operational  Test  and  Evaluation 
Agency. 

Between  January  and  May  1990,  we  attended  all  operational  tests  at 
White  Sands  Missile  Range,  New  Mexico,  and  the  majority  of  tests  and 
post-trial  meetings  at  Fort  Hunter-Liggett,  California.  We  discussed  the 
conduct  of  the  tests,  their  results,  and  analysis  with  numerous  test 
participants. 

We  reviewed  test  planning  documents  developed  by  the  adats  Program 
Manager,  the  Operational  Test  and  Evaluation  Agency,  other  Army 
organizations,  and  the  Director  of  Operational  Test  and  Evaluation  in 
the  Office  of  the  Secretary  of  Defense.  We  discussed  the  adats  program 
and  test-related  issues  with  Army  and  defense  intelligence  officials;  the 
Forward  Area  Air  Defense  System  Program  Executive  Officer;  the  adats 
Program  Manager;  Test  and  Experimentation  Command  test  conductors; 
and  officials  of  the  Air  Defense  Artillery  School  and  Center,  the  Opera¬ 
tional  Test  and  Evaluation  Agency,  the  Logistics  Evaluation  Agency,  the 
Army  Materiel  Systems  Analysis  Activity,  Headquarters  of  the  Depart¬ 
ment  of  the  Army,  and  the  Office  of  the  Secretary  of  Defense. 

We  examined  the  Army’s  internal  controls  for  ensuring  the  validity  of 
test  data  by  reviewing  the  instrumentation  and  audio,  video,  and 
manual  systems  for  trial  data  collection.  The  Army  recognized  that  the 
data  collection  system  would  contain  errors.  However,  due  to  time  con¬ 
straints,  we  did  not  test  those  systems  to  quantify  the  magnitude  of 
potential  errors. 


'The  act  a] so  requires  that  the  Director,  Operational  Test  and  Evaluation,  perform  a  similar  review. 
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Appendix  II 

Objectives,  Scope,  and  Methodology 


We  also  reviewed  internal  controls  for  ensuring  accuracy  in  data  anal¬ 
ysis  and  conducted  a  limited  test  of  those  controls.  On  the  basis  of  our 
observations  during  trials  and  attendance  at  post-trial  meetings,  we 
identified  anomalies  in  trial  conditions  and  conduct.  We  traced  selected 
anomalies  through  the  Army’s  data  validation  processes  to  determine 
how  Army  evaluators  had  treated  anomalies.  Due  to  the  complexity  and 
volume  of  generated  data  and  the  numerous  anomalies  identified  during 
the  trials,  we  did  not  quantify  the  impact  of  trial  anomalies  on  the 
Army’s  conclusions.  Further,  we  did  not  verify  the  Operational  Test  and 
Evaluation  Agency’s  programs.  Therefore,  our  current  analysis  accepted 
the  Army-developed  data  as  valid. 

We  conducted  our  work  from  December  1989  to  August  1990  in  accor¬ 
dance  with  generally  accepted  government  auditing  standards. 
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